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ABSTRACT 

The development of human-computer interaction (HCI) is a non-tangible way of communication between a human and a 
computer. Human hand gesture recognition (HGR) is one of the main topics in the area of computer science as an effective 
approach for HCI. Our project aims at developing a video player controlled by human hand gestures using the Convex Hull 
technique. This application uses the webcam to capture gestures by the user and perform basic operations such as play and pause. 
The gesture would serve as the direct command for operations such as play or pause the video based on the user’s gestures onto 
the screen. The project is implemented using Python, OpenCV. We can use our hand gestures to control certain functions of a 
computer like playing or pausing a video, increasing and decreasing volume, etc. 


Keywords - : HCI (Human Computer Interaction), Convex Hull, OpenCV, VLC controller, Webcam. 


I. INTRODUCTION 


Among many computer visions based interactive systems, 
designing hand gestures and facial expression-based 
Human-Computer-Interaction (HCI) systems remains to be 
a highly challenging task. Our main purpose is to find a non- 
tangible way to interact with the computer. Our project aims 
at developing a video player controlled by human hand 
gestures. The gestures would serve as the direct command 
for operations such as play or pause the video. It uses simple 
gestures to control the video. So, people don’t have to learn 
machine-like skills which are a burden most of the time, but 
against this people need only to recollect a group of gestures 
to control the video playback. In this paper, we have 
discussed a low system that uses hand gesture recognition 
technology to control the VLC media player. 


A. Aims and Objectives 


There are a lot of critical situations in the day-to-day lives 
of disabled people. We have come up with a small part of 
such life to make it easier using computer vision technology. 
In this paper, we have discussed a low system that uses hand 
gesture recognition technology to control the VLC media 
player. Among many computer visions based interactive 
systems, designing hand gestures and facial expression- 
based HCI system retains to be a highly challenging task. 
Our main purpose is to find a nontangible way to interact 
with the computer. 
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Our project aims at modifying the existing video player 
controlled by human hand gestures by making use of 
Convex Hull and OpenCV. 


Objectives: 


1) To minimize the use of keyboard and mouse in 
computer. 

2) To integrate gesture recognition features into any 
computer at a low cost. 

3) To help in the development of a non-tangible way 
to interact with the video player. 


B. Scope of the Project 


The Hand Gesture recognition is moving at tremendous 
speed for the futuristic products and services and major 
companies are developing technology based on the hand 
gesture system and that includes companies like Microsoft, 
Samsung, Sony and it includes the devices like Laptop, Hand 
held devices, Professional and LED lights. The use and 
adoption will become more cost effective and cheaper. It’s a 
brilliant feature turning data into features with mix of 
technology and Human wave. Smart phones have been 
experiencing enormous amount of Gesture Recognition 
Technology with look and views and working to manage the 
Smartphone in reading, viewing and that includes what we 
call touch less gestures. In the medical fields Hand Gesture 
may also be experienced in terms of Robotic Nurse and 
medical assistance. As the Technology is always revolving 
and changing the future is quiet unpredictable but we have 
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to be certain the future of Gesture Recognition is here to stay 
with more and eventful and Life touching experiences. 


C. Existing System 


Using Convex Hull and Gestures for Human-Computer 
Interaction (HCI) may be a pretty recent field of 
investigation. In that respect, only a couple of articles are 
published on that topic. The existing system that we are 
looking at is a video player controlled by hand gesture 
movements and postures. Development of Human- 
Computer Interaction (HCI) is a non-tangible way of 
communication between a human and computer. Our project 
aims at developing a video player controlled by human hand 
gestures using the Convex Hull concept. This application 
uses the webcam to capture gestures from the user and 
perform basic operations. 


Il. RELATED WORK 


As the aim was to use the Convex Hull concept to achieve 
the classification, Approach towards the necessity of 
Human-Machine Interaction using OpenCV along with 
Image acquisition, motion detection, thresholding, 
contouring, segmenting hand region effectively from a real 
time video sequence. A video stream is just a collection of 
frames at a particular time. So if we are successful in 
extracting those frames then it would be the initial step for 
our objective. These frames then get processed under image 
acquisition techniques using OpenCV. 


The developed model generates multiple channels of 
information from the input frames, and the final feature 
representation combines information from all channels. To 
further boost the performance, we propose regularizing the 
outputs with high-level features and combining the 
predictions of a variety of different models. We apply the 
developed Convex Hull concept code to recognize human 
actions in the real-world environment of the airport 
surveillance videos and they achieve superior performance 
in comparison to baseline methods. 


Understanding an image and classifying it into different 
hand gestures using Convex Hull, where we predict a class 
for each image. These images output in the form of an input 
to a modified VLC system then can allow the system to 
perform commanded actions. Analysing this gesture is 
useful in the following ways - 1. For a disabled individual 
who is unable to walk around in search of the system control 
devices (remote controller) or to press any keys on the 
system. 2. For an individual who is unable to understand the 
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upgraded skill of the controller (how to operate the 
controller). 3. For quick and efficient action on the video 
running in an emergency. 


II. DESCRIPTION 


A. Analysis 
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System Control 


Fig. 1 Basic Architecture of the Project 


Figure 1 is the basic structure that we plan to achieve 
where the simple human hand gestures will be considered as 
a control input for the video player to perform actions based 
on the appropriate mapping of the gestures along with their 
meaningful control actions (play, pause, volume up, volume 
down, resume). 


B. Feasibility Study 


1) Economic Feasibility: The cost of the project depends 
upon the accuracy of image capturing and pre-processing of 
those images using different techniques and finally the 
efficiency of the Convex Hull concept. Hence, by cost- 
benefit analysis we can conclude that benefit to cost ratio is 
not much high. 


2) Technical Feasibility: As mentioned earlier, the task of 
coding and debugging was made easier by using Convex 
Hull for the detection. As we use only a webcam to capture 
images, much GUI is not required, the app construction 
coding was also not a very difficult task. As every segment 
will be coded individually and separately, the risk 
assessment for each segment is easy. 
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3) Operational Feasibility: The user just has to upload a live 
video using a webcam. The image just has to be of high 
quality which can easily be captured from the webcam and 
processed. Hence, the controls are simple and basic. The 
efficiency of the project is based upon few factors, namely, 
the concept and its individual accuracy, the front end and its 
latency, the inter-connectivity. All these components can 
again be controlled individually and hence accuracy of the 
project as a whole can be easily controlled. 


C. Design 


Hand Gesture 


Capture input image from video 
Resize the image frame to 300 X 200 


Convert the RGB image into HSV colour space 


Binarise the hand image by thresholding & 


remove noise 





Detect boundary points and centroid of hand 
using convex hull 


xtract the Region Of Interest 





Find the Convexity defects (finger tips) 


Recognize the number of fingers 


Finger Count 


Fig. 2 Flowchart of the Image Detection 


The steps for Image Processing is been represented using 
Figure 2 which consists of eight steps such as capturing, 
resizing, converting, thresholding, contouring, extracting, 
detecting then finally recognizing. 


IV. IMPLEMENTATION 


Image Geatures System 


acquisition recognization control 





Fig. 3 Block Diagram 
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The design for the proposed system is shown in Figure 3 
The images and their masked labels are in .png format. The 
images are being captured from the live video stream and 
these images are further processed to achieve efficiency of 
the detection stage. In the detection stage, initially, contours 
are identified for the images and then these contours are used 
for the creation of the Convex Hull around the hand region. 
Then these image outputs are passed as control commands 
to the VLC media player to perform the associated action. 











Pixel values 
of R, G, B 


Fig. 4 Computer understanding Image 


As from figure 4 it is clear that, using three dimensions 
matrix pixel values (RGB values) for all the images in it 
crucial to distinguish between these images. Semantic 
Segmentation technique where the objective is to find 
different regions in an image and tag its corresponding 
labels, for this first step includes segmenting only the hand 
region from the live video stream and identify different 
possible gestures. The convex hull cluster is of peaks that 
cover the region of the hand. Here, we must clear the 
principle of the convex set, which means all lines between 
any 2 points within the hull are entirely within it. After 
determining the gesture, the specific functioning is 
performed. The method of recognizing the movement is a 
dynamic process. After operating the specific command 
from the gesture, go back to the initial step to accept other 
images to be processed, and so on. 





Fig. 5 Contours Convex Hull boundary 
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After this the gesture pictures were segregated into 
labelled folders as play, pause, volume up, volume down and 
resume appropriately. Exploratory Image details Analysis is 
an approach for analysing details related to images to 
Summarize their main characteristics, often with visual 
methods. Much complex visualization can be achieved with 
matplotlib and usually, there is no need to import other 
libraries. This is used for gaining a better understanding of 
data aspects like: 

-main features of data 

-variables and relationships that hold between them 
-identifying which variables are important for our problem 
Next we plan to input this processed images as input into the 
VLC media player and perform the necessary actions. 


V. RESULTS 


Depending on the contour defects and making use of 
OpenCV techniques the gestures are identified and classified 
into different gestures and there are visual threshold and 
color representations in the result. During the live video, 
streaming frames are extracted, processed, and converted 
into proper images along with region segmentation then 
these image outputs are supplied as input into the VLC video 
player to perform the associated action command related to 
VLC. Here we have implemented and restricted our project 
only for 4 gestures further scope would be to increase the 
number of gesture recognition and commands for the VLC 
video player. 





Fig. 6 Play Action using Hand’s Gesture 


The system is successful in detecting the hand gesture for 
play and recognizing the action to be performed, so the 
corresponding action of playing the video is functioned. 
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Fig. 7 Pause Action using Hand’s Gesture 


The system is successful in detecting the hand gesture for 
pause and recognizing the action to be performed, so the 
corresponding action of pausing the video is functioned. 





Fig. 8 Volume Down Action using Hand’s Gesture 


The system is successful in detecting the hand gesture for 
Volume down and recognizing the action to be performed, 
so the corresponding action of decreasing the volume for the 
video is functioned. 





Fig. 9 Volume Up Action using Hand’s Gesture 
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The system is successful in detecting the hand gesture for 
Volume up and recognizing the action to be performed, so 
the corresponding action of increasing the volume for the 
video is functioned. 





Fig. 10 Resume Action using Hand’s Gesture 


The system is successful in detecting the hand gesture for 
Resume and recognizing the action to be performed, so the 
corresponding action of resuming the video is functioned. 


VI. PLANS FOR FUTURE 


As per our plans, we have completed our project 
implementation. 
Next, we may aim to come with more gestures and their 
associated actions to be performed for building a completely 
modified VLC media player. A considerable amount of 
research yet needs to be done in this field as we need to take 
latency into consideration. The final aim is to reduce the 
project response time in real life as even a second delay 
could cause different control to be functioned. 


VII. CONCLUSION 


The gesture would serve as the direct command for 
operations such as play or pause the video based on the 
user’s gestures onto the screen. So people don’t have to learn 
machine-like skills which are a burden most of the time, but 
by contrast, people need only to remember a set of gestures 
to control the video playback. The Hand Gesture recognition 
1S moving at tremendous speed for the futuristic products 
and services and major companies are developing a 
technology based on the hand gesture system and that 
includes companies like Microsoft, Samsung, Sony and it 
includes the devices like Laptop, Handheld devices, 
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Professional and LED lights. Smartphones have been 
experiencing an enormous amount of Gesture Recognition 
Technology with look and views and working to manage the 
Smartphone in reading, viewing and that includes what we 
call touch less gestures. 
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